Papers with reasoning efficiency
AMO-Bench: Large Language Models Still Struggle in High School Math Competitions (2026.findings-acl)
Copied to clipboard
Junlin Liu, Shengnan An, Shuang Zhou, Dan Ma, Yehao Lin, Xinxuan Lv, Xuanlin Wang, Xiaoyu Li, Ziwen Wang, Xuezhi Cao, Xunliang Cai
| Challenge: | Existing benchmarks for mathematical reasoning are becoming less effective due to performance saturation. |
| Approach: | They propose to use a mathematical reasoning benchmark with Olympiad difficulty to evaluate top-tier LLMs. |
| Outcome: | The proposed benchmarks are cross-validated by experts to meet IMO difficulty standards and entirely original problems to prevent performance leakages from data memorization. |
Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication (2024.findings-emnlp)
Copied to clipboard
Weize Chen, Chenfei Yuan, Jiarui Yuan, Yusheng Su, Chen Qian, Cheng Yang, Ruobing Xie, Zhiyuan Liu, Maosong Sun
| Challenge: | Natural language (NL) has long been the predominant format for human cognition and communication, but its utility in LLMs has not been thoroughly examined. |
| Approach: | They propose to allow LLMs to choose the most suitable format before reasoning or communicating, and to automate the selection process. |
| Outcome: | The proposed format improves reasoning efficiency and reduces token usage while maintaining communicative effectiveness. |
START: Self-taught Reasoner with Tools (2025.emnlp-main)
Copied to clipboard
Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Bowen Yu, Binyuan Hui, Junyang Lin, Xiang Wang, Dayiheng Liu
| Challenge: | Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in complex reasoning through long chain-of-thought, yet they struggle with precise computations and algorithmic operations. |
| Approach: | They propose a training-free approach that activates LRMs’ latent tool-use capabilities through artificial hints and a framework that enables models to learn effective tool utilization through diverse hint patterns and rejection-based data synthesis. |
| Outcome: | Experiments show that START significantly improves state-of-the-art LRMs across challenging benchmarks, including competition-level mathematics (AMC23: 95.0%, AIME24: 75.6%) and graduate-level science questions (GPQA: 64.6%). |
Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce unnecessarily long explanations that reduce efficiency. |
| Approach: | They propose a length-aware reward that selectively penalizes insignificance tokens . they also propose 'dynamic length control' that encourages more detailed reasoning . |
| Outcome: | The proposed method reduces response length while maintaining correctness, the authors show . it selectively penalizes insignificance tokens while maintaining accuracy . |
Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve reasoning performance ignore the presence of unhealthy exploration that increases token usage without contributing to effective problem-solving. |
| Approach: | They propose an entropy-dynamics-aware prompt optimization framework that trains a lightweight optimizer to generate concise clarifications. |
| Outcome: | The proposed framework reduces ambiguity-induced early-stage uncertainty while preserving the model's reasoning capabilities. |
AutoL2S: Auto Long-Short Reasoning for Efficient Large Language Models (2026.findings-acl)
Copied to clipboard
Feng Luo, Yu-Neng Chuang, Guanchu Wang, Hoang Anh Duy Le, Shaochen Zhong, Hongyi Liu, Jiayi Yuan, Yang Sui, Vladimir Braverman, Vipin Chaudhary, Xia Hu
| Challenge: | Existing approaches to distilling large language models (LLMs) are inefficient and generate excessively long chain-of-thought reasoning even for inputs that admit concise solutions. |
| Approach: | They propose a distillation framework that empowers non-reasoning LLMs to think only when necessary. |
| Outcome: | The proposed framework reduces reasoning length up to 71% with minimal accuracy loss while preserving accuracy. |
MUR: Momentum Uncertainty guided Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Hang Yan, Fangzhi Xu, Rongman Xu, Yifei Li, Jian Zhang, Haoran Luo, Xiaobao Wu, Anh Tuan Luu, Haiteng Zhao, Qika Lin, Jun Liu
| Challenge: | Existing methods for optimizing reasoning quality are limited by overthinking. |
| Approach: | They propose a method that allocates thinking budgets to critical reasoning steps by tracking and aggregating step-wise uncertainty over time. |
| Outcome: | The proposed method reduces computation by over 45% on average while improving accuracy by 0.33–3.46%. |
Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Multi-domain machine translation (MDMT) is a unique challenge due to varying levels of linguistic complexity across domains. |
| Approach: | They propose a resource-rational framework that learns to modulate inference between intuitive and deliberate reasoning. |
| Outcome: | Evaluated on 15 benchmarks spanning in-domain and out-of-domain settings, as well as 3 seen and 59 unseen languages, with ablations across three backbone models, TwT-7B and Twt-14B outperform much larger SOTA reasoning models in translation quality, while reducing token usage by 32–60%. |
Exploration-Exploitation Reshaping towards Efficient Reasoning for Large Language Models (2026.findings-acl)
Copied to clipboard
Yufeng Shi, Weilin Luo, Yuxiang Zhang, Zongmeng Zhang, Haoyang Liu, Yubing Wang, Bin Wang, Wengang Zhou, Houqiang Li
| Challenge: | Large Reasoning Models (LRMs) are constrained by the overthinking issue. |
| Approach: | They propose a policy optimization framework that reshapes the exploration and exploitation through two core components: self-imitation and self-guidance exploration. |
| Outcome: | The proposed model achieves superior reasoning efficiency without compromising overall accuracy. |
Reasoning in a Combinatorial and Constrained World: Benchmarking LLMs on Natural-Language Combinatorial Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | NLCO evaluates large language models for combinatorial optimization (CO) . existing evaluations emphasize relatively simple reasoning competencies . |
| Approach: | They propose a combinatorial optimization benchmark that evaluates large language models on CO reasoning. |
| Outcome: | The proposed model can handle combinatorial optimization without writing code or calling external solvers. |
Thought-Action Graph Reasoning: Faithful and Efficient Reasoning of Large Language Models via Reusing Past Experience (2026.findings-acl)
Copied to clipboard
Zhixiao Qi, Feng Huang, Yunqi Zhang, Shijie Zhang, Qingqing Sun, Yongfeng Huang, Minghu Jiang, Shuai Chen, Tianyi Zhang
| Challenge: | Existing methods for integrating knowledge graphs with LLMs suffer from poor generalization or low reasoning efficiency. |
| Approach: | They propose a thought-action Graph (TAG) that decomposes LLM-KG interaction trajectories into fine-grained semantic operators and guides LLM to execute on them. |
| Outcome: | The proposed paradigm outperforms state-of-the-art methods on KGQA benchmarks while reducing the number of LLM calls and generated tokens. |
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry (2026.findings-acl)
Copied to clipboard
Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin, Hasan Muhammad Abdullah, Md Mehedi Hasan, Mohammad Zabed Hossain, Shifat E. Arman
| Challenge: | Visual question-based reasoning is a key component of vision-language models. |
| Approach: | They propose a framework for visual question-answering that integrates visual intent with visual severity to improve diagnostic accuracy. |
| Outcome: | The proposed framework improves diagnostic correctness, reduces hallucination, and increases reasoning efficiency. |
AdaMix: Adaptive Mixing for Short and Long Reasoning Adapters (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large reasoning models have improved efficiency but still face limitations such as conflicting objectives and limited adaptability. |
| Approach: | They propose an adaptive reasoning framework that applies a uniform, computation-intensive deep reasoning strategy to all problems. |
| Outcome: | The proposed framework reduces the average response length of DeepSeek-R1-Distill-Qwen-7B by 54.9% while improving accuracy by up to 4.8% on five mathematical datasets. |
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs (2026.findings-acl)
Copied to clipboard
Siyi Li, Jiajun Shi, Shiwen Ni, Ge Zhang, Shuaimin Li, Shijian Wang, Zhoufutu Wen, Yizhi LI, Hamid Alinejad-Rokny, Jiaheng Liu, Min Yang, Wenhao Huang
| Challenge: | Existing evaluations emphasize final accuracy or coarse token counts, and lack automated tools to separate essential logic from structural redundancy. |
| Approach: | They propose a graph-driven framework that quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path needed to reach a correct solution. |
| Outcome: | Evaluating 21 LRMs, the proposed framework quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path (SEP) needed to reach a correct solution. |